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Principals can play a key role in improving instruction and student achievement. The Institute of Education Sciences 
conducted a random assignment study of a professional development program for elementary school principals to support 
state and local efforts to improve school leadership. The program focused on helping principals conduct structured 
observations of teachers’ classroom instruction and provide targeted feedback. It provided nearly 200 hours of professional 
development over two years, half of it through individualized coaching. Key findings include: 


e Despite substantially increasing the amount of professional development principals received, the program did not affect 
student achievement or most teacher or school outcomes. For example, the professional development did not affect 
school climate or principal retention. 


e The program did not have the intended effects on principal practices that it targeted, which may explain its lack of 
effects on key student, teacher, and school outcomes. For example, it decreased the frequency of instructional support 
and feedback teachers received from principals, and it did not affect the number of teacher observations principals 


conducted or the usefulness of the feedback as reported by teachers. 
a | 


A number of studies suggest that principals can be critical to the success of their schools.' Perhaps because of this, the 
recent reauthorization of the Elementary and Secondary Education Act gives states and districts flexibility to use federal 
funds to support principal leadership. For example, states and districts can use these funds to provide principals with 
professional development on observing classroom instruction, providing feedback to teachers, and using evaluation 
results to inform decisions about teachers’ professional development and retention. 


Principal professional development programs use different approaches to improve principals’ performance, but there is 
limited research to guide their design and use. The only previous large-scale random assignment study of intensive 
principal professional development found that the McREL Balanced Leadership Program—which focuses on 21 leadership 
responsibilities—had limited effects on principals’ practices and no effects on student achievement." However, some less 
rigorous studies of the National Institute for School Leadership’s Executive Development program—which focuses on 
strategic thinking, coaching teachers, and driving and sustaining transformation—have found more positive effects." 


To expand the available evidence on ways to improve principals’ performance and student achievement, the U.S. 
Department of Education’s Institute of Education Sciences conducted a rigorous, large-scale evaluation of a specific 
approach to principal professional development. This approach sought to improve three areas of principals’ practices: 


1. Instructional leadership: Helping teachers improve instruction by observing teachers, using data from observations to 
provide feedback, and selecting curricula. 


2. Human capital management: Developing teachers and other staff by recruiting, managing, and retaining effective 
teachers and arranging professional development tailored to their needs. 


3. Organizational leadership: Developing the school as an organization by setting a school mission, improving school 
climate and culture, and deploying resources aligned with strategic goals. 


The Department competitively selected the Center for Educational Leadership (CEL) at the University of Washington to 
deliver the professional development program. At that time, CEL’s program was in wide use—it had implemented 
customized versions of its principal professional development in more than 100 school systems across nearly 30 states. 
Although CEL’s program included aspects of all three of the areas listed above, it emphasized instructional leadership. The 
program encouraged principals to conduct frequent classroom observations and document what teachers and students 
did and said in the classroom using a nonjudgmental, fact-based approach. An instructional framework and observation 
rubric guided principals’ observations and documentation.” The program asked principals to use this documentation to 
provide feedback to teachers. This emphasis was based on evidence suggesting the importance of principals using 
observation results and providing feedback to help improve the quality of teachers’ instruction.” (See Appendix A for more 
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details about the instructional framework, focus, structure, and implementation of the study’s professional development 
program.) 


The study’s principal professional development program was intended to improve principals’ practices in ways that would 
boost teacher effectiveness and school climate and, in turn, student achievement (Figure 1). It aimed to deliver a total of 
188 hours of professional development over two years, including: 


e  Anin-person summer institute to introduce all the principals participating in the program to CEL’s approach to 
instructional leadership (28 hours in the first year only); 


e _In-person group trainings in each district during the school year to provide principals with hands-on experience 
observing teachers and opportunities to discuss approaches to providing feedback to teachers (54 hours in the first 
year only); 


e Quarterly virtual professional learning community sessions for principals to meet with other principals and CEL 
coaches outside their district to discuss issues they were facing in their schools (6 hours in the first year only); and 


e Individualized coaching (both in-person and virtual) where principals worked with their CEL coach to identify areas of 
focus, set specific goals for those areas, implement strategies to address those goals, and analyze the effects of those 
strategies (100 hours total; 50 hours in each of the two years of the program). 


Figure 1. How the professional development for principals was intended to affect student achievement 
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Note: School climate is the extent to which students and teachers experience a safe, supportive environment with strong expectations 


for student and staff performance. Principal and teacher retention is the extent to which principals and teachers stay in their jobs 
for subsequent school years. 


The program was intended to supplement, rather than replace, any professional development that the district already 
provided to principals, although none of the districts offered principals intensive professional development. At the start of 
the study, coaches coordinated with districts to learn about any district-provided principal professional development and 
strategize about how their training and coaching could complement what districts were already doing. To connect their 
work with district priorities, coaches tried to use language that aligned with district and state standards and, where 
applicable, with the districts’ framework for conducting teacher observations. In addition, coaches kept district staff and 
principals’ supervisors informed about the program schedule and content. District staff did not help develop or deliver the 
program. 
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The study evaluated the program in 100 lower-performing elementary schools across eight districts (Exhibit 1). The study 
team randomly assigned schools to either participate in the program or not, but all principals continued to have access to 
any regular, district-provided professional development. By comparing the two sets of schools, the study can assess 
differences in the amount and types of professional development principals received and any effects expected to result 
from that, such as differences in principals’ practices, school climate, and student achievement. Multiple sources of data 
were collected to provide information about these outcomes, but principals’ feedback to teachers and the quality of 
teachers’ instruction were not directly observed.” The study examined outcomes for the two sets of schools in the two 
school years that the program was implemented—2015—2016 (Year 1) and 2016-2017 (Year 2). It also examined 
retention and student achievement in the year after the program was complete, 2017-2018 (Year 3), when principals 
could have continued to apply the lessons learned in their schools in ways that might have improved these outcomes. 


Exhibit 1. The study design 
Who participated? 


100 elementary schools from eight districts in five states: 100 principals, 3,266 teachers and 23,923 students 
Districts that were not already offering intensive professional development for principals volunteered to participate 
Compared with districts nationally, study districts were: 

- Larger (73 schools versus 7 in the average district nationally) 


- More concentrated in the South (five districts in the West South Central region and three districts in the South 
Atlantic region) 


Compared with schools nationally, study schools were: 
- Higher poverty (75 versus 55 percent of students eligible for free or reduced-price lunch) 


- Lower achieving (41st versus 50th percentile in their state in math and English language arts) 


Compared with elementary school principals nationally, study principals had fewer years of experience as school 
administrators (5 versus 7 years of experience, on average) “! 


How was the study conducted? 


Within each district, schools with similar characteristics were paired together before random assignment 


Within each pair, one school was randomly assigned to a group that participated in the study’s principal professional 
development program for two years, and the other school was assigned to a group that did not participate 


The two groups of schools had similar student achievement, student demographic characteristics, and educator 
characteristics at the beginning of the study 


The study compared outcomes for the two groups of schools to measure the effects of the program 


What data were used? 


To measure implementation: principal surveys, forms that documented principals’ participation in trainings and the 
extent to which trainings covered the intended content, and logs that coaches completed after each coaching session 
to document the focus and activities of the session 


To measure effects on principals’ practices and school climate: principal surveys, teacher surveys, and logs principals 
completed to document how they used their time during the school day 


To measure effects on student achievement: district records on students’ test scores in grades 3 through 5 for both 
years of program implementation plus one additional school year 


To measure effects on principal and teacher retention: district records on principals’ and teachers’ school 
assignments from district evaluation systems for both years of program implementation 
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THE STUDY’S PRINCIPAL PROFESSIONAL DEVELOPMENT PROGRAM DID NOT AFFECT 
STUDENT ACHIEVEMENT OR MOST SCHOOL OR TEACHER OUTCOMES MEASURED BY THE 
STUDY 


The program’s ultimate goal was to improve student achievement. To do so, it first aimed to improve principals’ practices, 
which in turn could affect school and teacher outcomes, including school climate and principal and teacher retention. The 
study examined the programs’ effects on these outcomes to learn about how well it worked in the study’s districts and 
schools. 


The professional development program did not affect student achievement during the two years of the program’s 
implementation or in the following year. In all three years of the study, average English language arts and math scores 
were nearly identical for students whose principals participated in the program and those whose principals did not. 
Average scores ranged from the 39th to 42nd percentiles across years and subjects (Figure 2). The program did not affect 
achievement in schools with less experienced principals (Appendix C, Table C.2). For schools with different levels of initial 
achievement, the program had some negative effects for schools with medium levels of initial achievement in the first 
two years and one positive effect for schools with lower initial achievement after three years (Appendix C, Table C.3)."" 


Figure 2. Student achievement, by year and participation in the professional development program 
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Source: Student administrative data on students in grades 3 through 5 in study schools in each year (23,299 students for English language arts and 
23,923 students for math in Year 1; 19,448 students for English language arts and 19,792 students for math in Year 2; 9,971 students for 
English language arts and 10,400 students for math in Year 3). 


Note: The Year 3 findings exclude three districts that did not provide data for that year. None of the differences is statistically significant at the 
.05 level, two-tailed test. 


Figure reads: At the end of Year 1, students in schools where principals participated in the program earned an average English language arts score at 
the 41st percentile in their state, and students in schools where principals did not participate in the program earned an average English language arts 
score at the 40th percentile in their state. 
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The professional development program generally did not affect school climate or principal and teacher retention. Principals 
and teachers in both groups of schools had similar perceptions of their school climate, rating it from 3.0 to 3.2, on 
average, on a 1- to 4- point scale where higher values reflect more positive perceptions (Figure 3). In addition, over a 
three-year period, the program did not improve the retention of the principals and teachers who worked in study schools 
before the study began (Figure 4). After one year of the program, schools that participated in the program had slightly 
lower teacher retention than schools that did not participate. However, there were no differences in teacher retention 
between these two groups of schools after two or three years. After three years, about half of the original principals and 
teachers remained in their schools, regardless of whether the principals participated in the program. 


Figure 3. Principal and teacher reports on school climate, by year and participation in the professional development 
program 
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Source: — Principal survey (90 principals in Year 1 and 92 in Year 2) and teacher survey (1,127 teachers in Year 1 and 1,095 in Year 2). 
Note: None of the differences is statistically significant at the .05 level, two-tailed test. 


Figure reads: In Year 1, principals who participated in the program gave their school climate a rating of 3.1 on a 1- to 4-point scale where higher 
values reflect more positive perceptions, compared with a rating of 3.2 for principals who did not participate in the program. 


4School climate, as reported by principals, includes the extent to which principals reported the school having problems with student absenteeism, 
widespread disorder in classrooms, and conflicts between students and teachers. The scale indicates whether each issue is a problem to a (1) great 
extent, (2) moderate extent, (3) small extent, or (4) not at all. 


5 School climate, as reported by teachers, includes the extent to which teachers reported cooperative effort among staff members in the school, the 
school administration being supportive and engaging, and not having problems with student misbehavior interfering with their teaching. The scale 
indicates whether teachers (1) strongly disagree, (2) disagree, (3) agree, or (4) strongly agree with statements about their school. 
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Figure 4. Principal and teacher retention, by year and participation in the professional development program 
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Source: — Principal administrative data (100 principals) and teacher administrative data (3,012 teachers). 


Note: Teacher administrative data were available for seven out of the eight districts. 


Figure reads: Among principals who led study schools in the baseline year, 84 percent of those who participated in the program were retained 


through the end of Year 1 compared with 86 percent of those who did not participate in the program. 


*Difference is statistically significant at the .05 level, two-tailed test. 
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THE PROFESSIONAL DEVELOPMENT PROGRAM HAD SOME NEGATIVE EFFECTS ON 
PRINCIPALS’ PRACTICES 


The program aimed to improve several principal practices. First, to increase the amount of instructional support that 
principals provided to teachers, it encouraged principals to observe instruction more often and provide teachers with 
more frequent feedback. To improve the quality of this instructional support, it sought to improve principals’ competence 
in providing feedback and the usefulness of the feedback they provided to teachers. To a lesser extent, the program also 
aimed to help principals better communicate plans for school improvement and select professional development that 
best met their teachers’ individual needs. The study examined the program’s effects on each of these practices. 


The professional development program did not affect the amount of time principals spent on instructional leadership. Both 
principals who participated in the program and those who did not spent similar amounts of time on most leadership 
practices. For example, in the second year, both groups spent fewer than 10 hours per week (25 percent of their time) 
evaluating instruction and providing feedback to teachers—instructional leadership practices that the program 
emphasized (Figure 5). Both groups spent nearly half of their time on organizational leadership responsibilities, such as 
student affairs (including attendance, discipline, and counseling) and administration (including operations and finance). 
Principals who participated in the program spent one extra hour per week on their professional growth, including time 
meeting with a coach or engaging in other professional development activities, and one less hour on curriculum, 
compared with principals who did not participate in the program. 


Figure 5. Hours per week principals spent on leadership activities, by content area and participation in the 
professional development program 
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Source: Principal time use logs (50 principals in Year 2). 


Figure reads: In Year 2, principals who participated in the program spent seven hours per week on the organizational leadership practice of student 
affairs, compared with eight hours per week for principals who did not participate in the program. 


* Difference is statistically significant at the .05 level, two-tailed test. 
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For each of the activities examined, both groups of principals spent their time in similar ways across both years, with a 
few exceptions (Appendix C, Table C.6). In the first year of the study, principals in the program worked more total hours 
per week (41 versus 38 hours). In the second year, both groups worked the same number of hours (41) per week. 


The professional development program did not affect the number of teacher observations that principals conducted and 
decreased the amount of instructional support and feedback teachers reported receiving from principals. Both groups of 
principals conducted about 20 observations in a typical classroom each year (Figure 6). These observations lasted about 
30 minutes (Appendix C, Table C.7). However, teachers whose principals participated in the program said they got less 
frequent instructional support and feedback compared with their counterparts whose principals did not participate in the 
program (Figure 6). In both years, teachers whose principals participated in the program received instructional support 
and feedback from their principal two fewer times, on average. 


Figure 6. Frequency of principals’ instructional leadership activities, by year and participation in the professional 
development program 
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Source: — Principal survey (78 principals in Year 1 and 90 principals in Year 2) and teacher survey (1,133 to 1,135 teachers in Year 1 and 1,100 to 
1,104 in Year 2). 


Figure reads: In Year 1, principals who participated in the program conducted 19 observations per classroom per year, compared with 20 
observations per classroom for principals who did not participate in the program. 
* Difference is statistically significant at the .05 level, two-tailed test. 


3This category includes classroom observations, feedback on teaching, developing specific instructional practice goals, using data to determine 
progress and suggest specific teaching actions, and other instructional supports. 


The professional development program had some negative effects on teachers’ perceptions of the quality of principals’ 
instructional support. In the two study years, both groups of principals on average rated the quality of their teacher 
observation skills 3.3 on a 1- to 4-point scale, where higher numbers reflect higher quality (Appendix C, Table C.8). Both 
groups of principals, on average, gave themselves identical ratings on their competence in providing instructional support 
in both years: 3.5 ona 1- to 4-point scale. Teachers in both groups generally had positive perceptions of principals’ 
instructional support and feedback (Table 1). However, teachers whose principals participated in the program had slightly 
more negative perceptions of some measures of principals’ support and feedback, particularly in the first year of 
implementation. The program did not affect teachers’ perceptions of the usefulness of the feedback received from 
principals in either year. Furthermore, the program did not increase the amount or quality of instructional support 
provided by less experienced principals or principals of schools with lower initial achievement (Appendix C, Table C.9). 
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Table 1. Teachers’ perceptions of the quality of principals’ instructional support, by year and participation in the professional 
development program 
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Source: Teacher surveys (791 to 1,124 teachers in Year 1 and 780 to 1,100 teachers in Year 2). 


Note: In the table, the effect of the program may not equal the difference between the means for schools that did and did not participate in the 
program in the table due to rounding. 


* Difference is statistically significant at the .05 level, two-tailed test. 


2“Principals’ competence in providing instructional support” includes whether teachers feel that principals know what effective teaching looks like; 
work directly with teachers to improve instruction; communicate clear standards for student learning and expectations for teacher performance; and 
encourage teachers to use what they learn from professional development, resources on teaching, and each other to improve instruction. The scale 
indicates whether teachers (1) strongly disagree, (2) disagree, (3) agree, or (4) strongly agree with statements about their school. 


5 “Usefulness of feedback received from principal” includes whether teachers feel that the feedback addressed pressing issues in their classroom, 
provided them with actionable steps for improvement, and helped them identify areas of instructional practice in which they need improvement. 
The scale indicates whether teachers felt the feedback met certain criteria (1) not at all, (2) to a small extent, (3) to a moderate extent, or (4) toa 
great extent. 


© “Usefulness of all types of instructional support from principal” includes whether teachers feel that the following types of instructional support 
were useful: classroom observations, feedback on teaching, developing specific instructional practice goals, using data to determine progress and 
suggest specific teaching actions, and other instructional supports. The scale indicates whether teachers felt the instructional support met certain 
criteria (1) not at all, (2) to a small extent, (3) to a moderate extent, or (4) to a great extent. 


, 


The professional development program had few effects on the principal practices that were not its primary focus. Principals 
human capital management practices, measured by the frequency, intensity, and content of professional development 
that they arranged for teachers, did not change. The program also had few effects on principals’ organizational leadership 
practices, measured by principals’ and teachers’ perceptions of the frequency and coherence of plans for school 
improvement (Appendix C, Table C.10). 
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THE PROGRAM INCREASED THE AMOUNT OF PROFESSIONAL DEVELOPMENT THAT 
PRINCIPALS RECEIVED 


Given that the professional development program did not affect some principal practices and negatively affected others, 
it is useful to examine the program’s implementation, how it added to the support principals were already receiving, and 
the content that it emphasized. 


Principals received the planned amount of professional development. Over two years, principals received all 188 hours 
planned for the program. In the first year, they received 134 of the 138 hours planned (Figure 7). In the second year, they 
received more than planned—54 hours, rather than 50 hours. Nearly all principals—84 percent—stayed in their schools 
and participated in the program for the full two years. In 8 schools (out of a total of 50), the original principal left, and the 
program was offered to the new principal. (One school did not participate in the program in the second year.) 


Figure 7. Planned and actual amounts of professional development for principals participating in the program 
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Sources: Participation forms (14 total forms in Year 1); observation forms (12 total forms in Year 1); and coaching logs (50 principals in Year 1 and 
49 principals in Year 2). 


Figure reads: In Year 1, the study planned for principals to receive 50 hours of individualized coaching, 28 hours of professional development from 
the summer institute, 54 hours from group trainings during the school year, and 6 hours from the professional learning community sessions—a total 
of 138 hours across the four components. 


Consistent with its intended focus, the professional development program emphasized instructional leadership. Across all 
four components of the program, 70 percent of the professional development time focused on instructional leadership. 
Human capital management and organizational leadership received much less attention (16 and 14 percent, respectively) 
(Appendix A, Table A.2). In addition, principals worked with their coaches to set goals for their time together—all 
principals set at least one goal in the area of instructional leadership, compared with only about half who set at least one 
goal in the area of human capital management or organizational leadership (Figure 8). 
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Figure 8. Focus of coaching goals for principals participating in the professional development program 
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Source: Coaching logs (50 principals in Year 1 and 49 principals in Year 2). 


Figure reads: Across the two years of implementation, 100 percent of principals who participated in the program had at least one principal-focused 
coaching goal in the instructional leadership area. 


The program increased the number of total hours of professional development that principals received. The professional 
development program was intended to add to, rather than replace, any professional development that the district already 
provided to principals. In the first year, those participating in the program got a total of 100 hours more professional 
development than other principals (220 versus 120 hours, Figure 9). The extra training hours included more hours of 
group training, one-on-one development, and other development, consistent with the program’s inclusion of group 
training, coaching, and professional learning communities. In the second year, principals in the program had more hours 
of one-on-one development (58 versus 19 hours) than other principals. This finding is consistent with the fact that the 
program provided only individualized coaching, and not group training, in the second year. 
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Figure 9. Overall number of hours of professional development that principals received, by year and assignment to 
the program 
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Source: Principal survey (90 principals in Year 1 and 90 to 92 principals in Year 2). 

Figure reads: In Year 1, principals who participated in the program received 19 hours of professional development from a formal degree program or 
course, compared with 7 hours for principals who did not participate in the program. 

*Difference is statistically significant at the .05 level, two-tailed test. 


aThis category includes mentoring and coaching. 


The program increased the percentage of principals who received professional development related to instructional 
leadership. Consistent with the program’s emphasis on instructional leadership, more than 90 percent of principals who 
participated in the program reported receiving professional development related to (1) observing classroom instruction 
and (2) providing feedback to teachers on their instruction, compared with about 70 percent of principals who did not 
participate in the program (Appendix C, Table C.12). 
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DIRECTIONS FOR FUTURE RESEARCH ON PRINCIPAL PROFESSIONAL DEVELOPMENT 


The study’s principal professional development program was based on the theory that improving principals’ leadership 
practices should improve school, teacher, and student outcomes. Although implemented as intended, the program did 
not improve these outcomes, on average. However, impacts were positive for some schools and negative for others 
(Appendix C, Figures C.1-C.6). Analyses exploring these differences may provide information about this theory for 
improving outcomes. They could also suggest potential avenues for improving principal professional development that 
future studies could explore. 


First, the study provides suggestive evidence that professional development focused on principals’ instructional 
leadership could help improve student achievement. The program’s effects on certain instructional leadership practices 
were positively associated with its effects on student achievement (Appendix C, Tables C.13—C.15). These practices 
included principals’ competence in providing instructional support, the frequency of principals’ instructional feedback 
and support, and the usefulness of teachers’ interactions with principals about instruction. However, these findings are 
only suggestive, because principals who experienced larger and smaller effects from the program may differ on other 
characteristics that could be related to student achievement in their school, such as prior leadership ability. 


Second, the study highlights various challenges to effectively changing principals’ leadership practices. Principals in the 
study received the intended amount of group trainings and individualized coaching that included opportunities to 
practice and discuss the new skills promoted. However, the program did not improve principals’ practices, on average. 
Spending higher percentages of coaching time on activities that might be expected to improve principals’ practices— 
such as hands-on activities, developing plans to address principals’ specific problems, or instructional leadership 
activities—was not related to the program’s effects on leadership practices (Appendix C, Tables C.16 and C.17). These 
findings are also only suggestive, because principals who spent more and less time on each of these activities may differ 
in terms of their schools’ needs and other characteristics. 


The study highlights issues for future studies to explore. Although the study did not directly observe the quality of 
principals’ instructional support or teachers’ instruction, the program did not improve teachers’ perceptions of the 
usefulness of principals’ feedback. Future studies could investigate the effectiveness of different professional 
development activities for improving principals’ feedback to teachers. In addition, despite a focus on classroom 
observations and teacher feedback, the program did not increase the number of observations that principals 
conducted. In fact, principals spent less than a quarter of their work week evaluating instruction and providing 
feedback to teachers. Future research could explore how principal professional development programs focused on 
instructional leadership could help principals make more time for these instructional leadership activities. 
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ENDNOTES 


' Hallinger and Heck 1998; Harris et al. 2010; Knapp et al. 2006; Leithwood et al. 2004 

Jacob et al. 2014 

i Nunnery et al. 2010, 2011, 2016 

Principals could either use CEL’s 5 Dimensions of Teaching and Learning (5D)™ rubric or the one used by their district, 
such as the Framework for Teaching (Danielson 2009). 

’ Clark et al. 1980; Grissom et al. 2013 

“ The study team intended to use ratings of teachers from their districts’ evaluation systems to examine teacher 
effectiveness. However, because these data were unavailable for nearly half of the teachers in the study schools, the 
study did not analyze these data. 

“ll Taie and Goldring 2017 


“ll Throughout this report, effects are designated as statistically significant if the p-value is less than 0.05. Appendix D 
presents an alternative approach to interpreting study findings that does not rely on p-values or statistical significance. 
This alternative approach avoids the common misinterpretation of statistical significance (p-value < 0.05) as meaning that 
there is at least a 95 percent chance that an intervention had an effect (Greenland et al. 2016). 
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